Welcome back to deep learning. So today we want to discuss a couple of more application

oriented topics. We want to look into image processing and in particular into segmentation

and object detection. So let's see what I have here for you. Here's the outline of the

next five videos and we will first introduce the topic of course. Then we'll talk about

segmentation. So we'll motivate it and discuss where the problems are with segmentation.

And then we want to go into several techniques that allow you to do good image segmentation.

You will see that there is actually very interesting methods that are super powerful and can be

applied on a wide variety of tasks. After that we want to continue and talk about object

detection. So this is a kind of related topic. And with object detection we then want to

look into different methods how you can find objects and scenes and how you can actually

identify which object belongs where. So let's start with the introduction. So far we looked

into image classification essentially. And here you can see that the problem is that

you simply have the classification to cat but you can't make any information out of

the spatial relation of objects to each other. An improvement is image segmentation. So in

semantic segmentation you then try to find the class of every pixel in the image. So

here you can see in red that we marked all of the pixels that belong to the class cat.

Now if we want to talk about object detection we have to look into a slightly different

direction. So here the idea would be to identify essentially the area where the object of interest

is and you can already see here if you use for example the methods that we learned in

visualization we will probably not be very happy because we would simply identify pixels

that are related to that class. So this has to be done in a different way because we are

actually then interested in finding the different instances. So we want to be able to figure

out different cats in a single image and then find bounding boxes. So this is essentially

the task of object detection and instance recognition. Now lastly when we have mastered

those two ideas then we also want to talk about the problem of instant segmentation.

So here it's not just that you find all pixels that show cats but you actually want to differentiate

different cats and assign the segmentations to different instances. So this is then instant

segmentation which will be in the last video about these topics. So let's go ahead and

talk a bit about ideas towards image segmentation. Now in image segmentation we want to find

exactly which pixels belong to that specific class and we want to delineate essentially

the boundary of meaningful objects. So all of these regions that are within the boundary

should have the same label and they belong to the same category. So each pixel gets a

semantic class and we want to generate a pixel-wise dense labeling. So these concepts are of course

here shown on images but technically you can also do similar things on sound when you for

example look into spectrograms. So the idea in images would be that we want to make out

of the left hand image the right hand image and you can see already that we find the region

that is identified by the airplane here and we find the boundary. Of course this is a

more simple task here you can also think about more complex scenes like this example here

from autonomous driving and here we are for example interested in where the street is,

where persons are, where pedestrians, where are vehicles and so on and we want to mark

them in this complex scene and similar tasks can also be done for medical imaging for example

if you are interested in identification of different organs, where the liver is, where

the vessels are or where cells are. So of course there are many many more applications

that we won't talk here about. There's aerial images if you process satellite images of

course also in robotics and also image editing where you can show that these kind of techniques

have very useful properties. So of course if we want to do so we need to talk a bit

about evaluation metrics and of course we have to be somehow able to measure the usefulness

of a segmentation and this depends then on several factors of course the execution time,

memory footprint and of course the quality and the quality of a method we need to assess

with different metrics. The main problem here is that very often the classes are not equally